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Abstract 

This paper is concerned with providing the maximum principle for a 
control problem governed by a stochastic evolution system on a separa- 
ble Hilbert space. In particular, necessary conditions for optimality for 
this stochastic optimal control problem are derived by using the adjoint 
backward stochastic evolution equation. 
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1 Introduction 

Consider a stochastic controlled problem governed by the following stochastic 
evolution equation (SEE): 

dX(t) = (AX(t) + b(X(t), v(t)))dt + a(X(t),u(t))dW(t), t e [0, T], 
X(0) = x . 

We shall be interested in trying to minimize the cost functional, which is given 
by equation (12. 2 j) below, over a set of admissible controls. 

This system is driven mainly by a possibly unbounded linear operator A on a 
separable Hilbert space H and a cylindrical Wiener process W on H. Here u(-) 
denotes a control process. 

We shall derive the maximum principle for this control problem. More pre- 
cisely, we shall concentrate on providing necessary conditions for optimality for 
this optimal control problem, which gives this minimization. For this purpose we 
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shall apply the theory of backward stochastic evolution equations (BSEEs shortly) 
as in equation (13. 2p in Section [3j which together with backward stochastic differ- 
ential equations (BSDEs) have become of great importance in a number of fields. 
In [10], [13], p2], p3], [3], @] and [16] one can find applications of BSDEs to 
stochastic optimal control problems. Some of these references have also studied 
the maximum principle to find either necessary or sufficient conditions for opti- 
mally for SDEs or SPDEs. Necessary conditions for optimality of the control 
process u(-) and its corresponding solution but for the case when a does not 
depend on v{-) can be found in [TO] . 

In our work here we allow a to depend on the control variable and study a 
stochastic control problem associated with the former SEE. This control problem 
is explained in details in Section |2j and the main theorem is stated in Section [3] 
and proved together with all necessary estimates in Section HI We emphasize 
that our work here does not need go through the technique of Hamilton- Jacobi- 
Bellman equations nor the technique of viscosity solutions. We refer the reader 
to [5] for this business and to [6] and some of the related references therein for 
the semi-group technique. 

On the other hand, sufficient conditions for optimality for this optimal control 
problem can be found in [I]. We refer the reader also to [3]. 

2 Statement of the problem 

Let (fl, J 7 , P) be a complete probability space and denote by M the collection of P- 
null sets of J 7 . Let {W(t), < t < T} be a cylindrical Wiener process on H with 
its completed natural filtration T t = a{£ o W(s) , < s < t , i e H*} V A/", t > 0; 
see [Tj for more details. 

For a separable Hilbert space E denote by L^-(0,T; E) to the space of all 
{Ft, < t < T} - progressively measurable processes / with values in E such that 




This space is Hilbert with respect to the norm 




Moreover, if / e L^(0, T; L 2 (H)), where L 2 (H) is the space of all Hilbert-Schmidt 
operators on H, the stochastic integral f f{t)dW{t) can be defined and is a con- 
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tinuous stochastic martingale in H. The norm and inner product on L,2(H) will 
be denoted respectively by || • || 2 and (•, -) 2 - 

Let us assume that O is a separable Hilbert space equipped with an inner 
product -) , and U is a convex subset of O. We say that u(-) : [0, T] x O — > O 
is admissible if //(•) G L?p{Q,T;G) and u(t) G U a.e., a.s. The set of admissible 
controls will be denoted by li a d- 

Suppose that b : H x O — > H and a : H x O — >■ L 2 (H) are two continuous 
mappings, and consider the following controlled SEE: 

I ^(0)=x , lAiJ 

where u(-) G W a rf. A solution (in the sense of the following theorem) of (12. ip will 
be denoted by X u ^ to indicate the presence of the control process u(-). 

Let £ : H x O — > M. and (p : H — > R be two measurable mappings such that 
the following cost functional is defined: 

J(i/(.)):=E[ [ T e(X^(t),v(t))dt + <f>{X"U{T))}, v{-)EU ad . (2.2) 

For example one can take £ and to satisfy the assumptions of Theorem 13.21 in 
Section |3J 

The optimal control problem of the system (12. ip is to find the value function 
J* := inf{J(i/(-)) : is(-) G tl a d\ and an optimal control v*(-) G tl a d such that 

J* = J(v*(-)). (2.3) 

If this happens, the corresponding solution X u *^ is called an optimal solution of 
the stochastic control problem (l2.ip -f l2T3"|) and (X u *^ , is called an optimal 

pair. 

We close this section by the following theorem. 

Theorem 2.1 Assume that A is an unbounded linear operator on H that gener- 
ates a Co-semigroup {S(t), t > 0} on H, and b,a are continuously Frechet dif- 
ferentiable with respect to x and their derivatives b x , o~ x are uniformly bounded. 
Then for every z/(-) G U ac i there exists a unique mild solution X u ^ on [0, T] to 
\2.1\) . That is X u ^ is a progressively measurable stochastic process such that 
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X(0) = x and for all t G [0, T] , 

X v{ -\t) = S{t)x + [ S(t-s)b(X"U(s),v(s))ds 

Jo 

+ [ S(t- s)a{X^{s),u{s))dW{s). (2.4) 
Jo 

The proof of this theorem can be derived in a similar way to those in pO, Chapter 
7] or [11]. 

From here on we shall assume that A is the infinitesimal generator of a Co- 
semigroup {S(t), t > 0} on H. Its adjoint operator A* : V(A*) C H ->• H 
is then the infinitesimal generator of the adjoint semigroup {S*(t), t > 0} of 
{S(t),t>0}. 

3 Stochastic maximum principle 

It is known from the literature that BSDEs play a fundamental role in deriving 
the maximum principle for SDEs. In this section we shall search for such a role 
for SEEs like (12.11) . To prepare for this business let us first define the Hamiltonian 
by the following formula: 

H: H xOx H x L 2 (H) — > K, 

H(x, v, y, z) := £(x, v) + (b(x, u),y) H + {cr(x, u), z) r (3.1) 

Then we consider the following BSEE on H: 

' -dyW(t) = (A*Y<\t) + V x U(X^\t),v(t),Y^\t),Z^(t)))dt 

-Z»U(t)dW{t), 0<t<T, (3.2) 

y<.-)(T) = V0(X^)(T)), 

where V0 denotes the gradient of (ft, which is defined, by using the directional 
derivative D<j)(x)(h) of at a point x G H in the direction of h G H, as 
(V<f>(x),h} H = D(f>(x)(h) (= 4> x {h)). This equation is the adjoint equation of 

(JO). 



As in the previous section a mild solution (or a solution) of (J372]) is a pair 



Necessary conditions for optimality for SEEs 



5 



(Y, Z) G Ljr(0, T; iJ) x L^(0, T; L 2 {H)) such that we have P- a.s. for all t G [0, T] 
y(0( t ) = s*(T-t) V(j){X v{ -\T)) 



Theorem 3.1 Assume that b,a,£,(f) are continuously Frechet differentiable with 
respect to x, the derivatives b x ,a x ,a u ,£ x are uniformly bounded, and 



for some constant k > 0. 

Then there exists a unique solution (Y u( -'\ Z v ^) of the BSEE 

The proof of this theorem can be found in [2] or [9]. An alternative proof by 
using finite dimensional framework through the Yosida approximation of A can 
be found in [T5] . 

Our main result is the following. 

Theorem 3.2 Suppose that the following two conditions hold, 
(i) b,a,£ are continuously Frechet differentiable with respect to x,u, (p is contin- 
uously Frechet differentiable with respect to x, the derivatives b x , b u , a x ,a u ,£ x , l v 
are uniformly bounded, and 



for some constant k > 0. 

(ii) i x is Lipschitz with respect to u uniformly in x. 

If (X* is an optimal pair for the control problem Ii2.1\) - I[27^) . then there 
exists a unique solution (Y u *('\ Z u *('') to the corresponding BSEE ( tff.ffj) s.t. the 
following inequality holds: 



H{X v ^\t),v*{t),Y v ^\t),Z v ^\t)) < H(X u ^\t),u, Y u *U(t), Z v *^\t))) 

a.e. t G [0,T], a.s.M v G U. 




\4>x\l(h,h) <k(l+ \x\ H ) 



\4>x\l(h,h) <k(l + \x\ H ) 



To illustrate this theorem let us present an example. 
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Example 3.3 Let H and O be two separable Hilbert spaces as considered earlier, 
and let U = O. We shall study in this example a special case of the control problem 
Ii2.1\) - li2.3\) . In particular, given <fi as in Theorem \3.2l we would like to minimize 
the cost functional: 

J(u(-))=E[ F \v{t)\ldt\+n<l>{X^\T))\ (3.4) 



subject to: 

dX v U(t) = (AX u U(t) + Bv(t))dt+ Du(t)dW(t), t e (0,T\, 
X v U(0) = x o eH i {6 - b) 

where B is a bounded linear operator from O into H and D is another bounded 
linear operator from O into L 2 (H). 

The Hamiltonian is then given by the formula: 

U(x, v, y, z) = \u\ 2 + (B v , y) R + (Du , z) L2[H) , 

where (x, v, y , z) G H x O x H x L 2 (H), and the adjoint BSEE is 



dY u ^{t) = A* Y u U(t)dt - Z v ^(t)dW(t), t g [0, T), 
Y»(-)(T) = V0(X y W(T)). 



(3.6) 



From the construction of the solution of ( Iff. 6\) . as e.g. in [2, Lemma 3.1], this 
BSEE attains an explicit solution: 

Y^\t) = E [ S*{T - t) V0(X"W(T)) | T t ], 

Z<\t) = S*(T-t) R u{ -\t), 
where R u ^ is the unique element of L^r(0, T; L 2 {H)) satisfying 

V0(X" w (T)) = E [ V<t>{X"U(T)) ] + / if dW{t). 

Jo 

On the other hand, for fixed (x,y,z), we note that the function 
v I—)- T-L(x, v, y, z) attains its minimum at v = ~ [B* y + D* z) (g U), where 
B* : H —7- O and D* : L 2 (H) — > O are the adjoint operators of B and D respec- 
tively. So we elect 

v*{t,u)= - [B* Y v * (t, uj) + D* Z v * (t, u)) (3.7) 
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as a candidate optimal control. 

It is easy to see that with these choices all the requirements of Theorem \3.S\ 
are verified. Hence this candidate u*(-) given in flff. 7| ) is an optimal control for 
the problem ^3.^ - ^3. 5\) . and it corresponding optimal solution X* is the solution 
of the following SEE: 

dX*(t) = (AX*(t)+ \ B{B*Y*(t) + D* Z*{t))\dt 

+ \ D (B*Y*(t) + D* Z*(t))jdW(t), t G (0,T], 

X*(0) =x . 
Finally, the value function attains the formula 
1 f T 

J* = -E[ / \B*Y*(t) + D*Z*(t)\ 2 dt]+E[cf)(X*(T))}. 
4 Jo 

Remark 3.4 A concrete example in the setting of Example VS. 3\ can be constructed 
by taking H = O = L 2 (R d ), d > 1, A = |A (half-Laplacian), B = id H , Dv := 
(v , fi) H Q 1 / 2 , cj)(x) = (p,x) H , for some fixed elements h, p of H and a positive 
definite nuclear operator Q on H. 

The computations now ofH,Y*,Z*, u* , X* become direct from the correspond- 
ing equations in Example \3.3\ 

4 Proofs 

Let !/*(•) be an optimal control and X u *^ be the corresponding solution of ( 12. ip . 
Let u(-) be an element of Lj-(0,T; O) such that u*(-) + v{-) G U a d- For a given 
< e < 1 consider the variational control: 

u e (t) = u*(t)+ev(t), te [0,T]. 

We note that the convexity of U implies that u £ (-) G U a d- Considering this control 
v e (-) we shall let X Ve ^ be the solution of the SEE (12.11) corresponding to t / £ (-), 
and denote it briefly by X e . 

Let p be the solution of the following linear equation: 

dp{t) = (Ap{t) + b x {X*{t),v*{t))p{t) + b v {X*{t),v*{t))v{t))dt 

+ ( a x (X*(t), u*(t)) p(t) + a u (X*(t), u*(t)) u(t))dW(t), (4.1) 

p(0) = 0. 
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The following three lemmas contain estimates that will play a vital role in 
deriving our main variational equation and the maximum principle for our control 
problem. 

Lemma 4.1 Assume condition (i) of Theorem \3.2[ Then 

sup E[\p{t)\ 2 ] < oo. 
te[o,T] 

Proof. The solution of ( 14. ip is given by the formula 

p(t) = f S(t - s) (b x (X*(s),u*(s)) p(s) + h(X*(s),v*(s)) u(s))ds 
Jo 

+ / S(t-s)(a x (X*(s),v*(s))p(s) + a„(X*(s),v*(s))is(s))dW(s). (4.2) 
Jo 

By using Minkowski's inequality (triangle inequality), Holder's inequality, 
Burkholder's inequality for stochastic convolution together with assumption (i) 
and Gronwall's inequality we obtain easily 

sup E[\p(t) | 2 ] < C (4.3) 
te[o,T] 

for some constant C > 0. ■ 

Lemma 4.2 Assuming condition (i) of Theorem \3.2\ r we have 

sup E[\X £ (t)-X*(t)\ 2 ] = 0(e 2 ). 

*6[0,T] 

Proof. Observe first from ( 12.41) that 

X £ (t) - X*(t) = [ S(t - s) ( b(X E , is £ (s)) - b(X*(s),u*(s)) )ds 
Jo 

+ / S(t-s)(a(X £ ,p £ (s))-a(X*(s),iy*{s)))dW{s). (4.4) 
Jo 

Hence 

E [ \X £ (t) - X*(t)\ 2 ] < 2M 2 T E [ / | b(X £ (s),v £ (s)) - b(X*(s), u*(s)) \ 2 ds] 

Jo 

+ 2M 2 E[[ \\a(X £ ,u £ (s))-a(X*(s),u*(s))\\ 2 2 ds}. (4.5) 
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where M := sup \\S(t)\\ L ( H ,H)- 
te[o,T] 

Secondly, from condition (i) we get 



t 

2 



E[ / \b(X £ (s),v £ (s)) -b(X*(s),v*(s))\ 2 ds] 
Jo 

< 2E[[ \b(X £ (s),u £ (s))-b(X*(s),u £ (s))\ 2 ds} 

Jo 

+ 2E[ / \b{X*(s),v £ {s))-b{X*{s),iy*(s))\ 2 ds] 
Jo 

= 2E[[ \~b x (s,e)(X £ (s)-X*(s))\ 2 ds} + 2E[ / \6 £ b(s)\ 2 ds] 
Jo Jo 

< 2CiE[ / \X £ (s) -X*(s) \ 2 ds] + 2C 2 e 2 , (4.6) 



where, for y G H, 



b x (s,e)(y)= [ b x {X*(s) + e(X £ {s)-X*(s)),v £ {s)){y)d9, 
Jo 

5 £ b(s) = b(X*(s),u £ (s)) - b(X*(s), v*(s)), 

C\ is a positive constant, and C2 is another positive constant coming thanks to 
(i) from the following inequality: 

E [ [ T I 6 £ b(s) | 2 ds } = E [ [ T I b(X*(s), u £ (s)) - b(X*(s), u*(s)) \ 2 ds } 



Jo 

T r l 

2 



= E[ I / K(X*(s),v*(s) + 6(v £ (s)-u*(s)))(v £ (s)-v*(s))de\ 2 ds] 
Jo Jo 

<C 2 e 2 . (4.7) 

Similarly, 

E [ f I o{X £ {s)Ms)) ~ o-(X*(s),v*(s)) | 2 ds] 
Jo 

<2C 3 E[ f \X e (s) -X*(s) \ 2 ds] + 2C 4 e 2 , (4.8) 
Jo 

for some positive constants C3, C4. 

Finally, by applying (14. 6p . (14.81) in (14.41) and then using Gronwall's inequality 
we find that 

E[\X £ (t)-X*(t)\ 2 }<C 5 s 2 (4.9) 
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for some constant C5 > that depends in particular on Cj, i = 1, ... ,4 and M. 
Hence the proof is complete. ■ 

Keeping the notations b x and S £ b used in the preceding proof we state the 
following lemma. 

Lemma 4.3 Let r) £ (t) = W _ p(t). Then, under condition (i) of Theo- 

remlEM 

lim sup E[\r) £ {t)\ 2 } = 0. 
£ -> 0+ te[o,T] 

Proof. From the corresponding equations (12.11) and (14. ip we deduce that 

Ve(t) = 

[ S(t-s)[- (b(X £ (s),v £ (s)) - b(X*(s),u £ (s))) - b x (X*(s), v*(s))p(s) ] ds 
Jo £ 

+ / S(t-s)[-5 £ b(s)-b u (X*(s),u*(s))u(s)]ds 
Jo £ 

+ f S(t-s)[- (o-(X £ (s),v £ (s)) 
Jo £ 

- <r{X*(s), u £ {s))) - <t x (X*{s),v*{s))) p{s) } dW(s) 

r* 1 

+ / S(t-s)[-5 £ o-(s)-a„(X*(s),p*(s))v(s)]ds 
Jo 6 

= [ S(t-s)[b x (s,e)r ]£ (s) + (b x (^e)-b x (X*(s),u*(s)))p(s)]ds 
Jo 

f* 1 

+ / S(t-s)[-S e b(s)-K(X*(s),u*(s))u(s)]ds 
Jo £ 

+ / S(t - s)[dr x (s,e) V£ (s) + (a x (s,e) - a x (X*(s),u*(s))p(s)]dW(s) 
Jo 

+ / S(t-s)[-5 £ a(s)-a u (X*(s),u*(s))u(s)]dW(s), (4.10) 
Jo 6 

where 

5 £ a(s) = a(X*(s), u £ (s)) - a(X*(s), u*(s)) 

and 

a x (s,e)(y) = [ cr x (X*(s) + 6{X £ {s) - X*(s)),v s (s))(y)d6, y E H. 
Jo 
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Consequently, from (i) and as in the proof of Lemma 14.21 it follows that 

E[|^(t)| 2 ] <C 6 f E[\ Ve (s)\ 2 ]ds + p(e), 
Jo 

for all t G [0,T], where 

p{e) = 8MTK [ f T | (b x (s,e) — b x (X*(s), v*(s)) ) p(s) | 2 c?s ] 
Jo 

+ 8ME[ / || (a x (s,e) - a x (X*(s),u*(s)))p(s)\\lds} 
Jo 



(4.11) 



+ 4MTEI 



4ME 



- 5 £ 6(s) - !/*( S )) !/( S ) | 2 ds] 

- 5 £ <r(s) - a u (X*{s),p*(s))p(s)\\lds). (4.12) 



But (i), ( 14. 3 ft and the dominated convergence theorem give 

E [ f | ( 6 2 ( S , e) - v*{s)) ) p(s)\ 2 ds] 

Jo 

= E[/ |/ (^(X*( S ) + e(X e ( S )-X*( S ))^ e ( S )) 
Jo Jo 



< 



f [ E[\(b x (X*(s) + 9(X £ (s)-X*(s)),u e (s)) 
Jo Jo 

-b x (X*(s),v*(s)))p(s)\ 2 ]d6ds 



-> 0, as e -> H 



Similarly we have 

E[ / T || (cr a ( S ,e) - tr x (X*(s)) ||| rfs ] 0, 
Jo 

as e — > + . 

On the other hand, as done for (14. 7p . 

E [ f | - 5 £ b(s) - b v {X*{s), u*{s)) v{s) | 2 ds ] 
Jo £ 

" X(X*(8),u*( a ) + 9(u e (8)-u*(a))) 




< I I E 

'o Jo 



-^pr(s),*/( S )) 





2 - 







d6 ds -> 0, 



(4.13) 



(4.14) 
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if e + , by using (i) and the dominated convergence theorem. Similarly, 

E[l ||- S e a(s) -a v {X*{s),v*{s))u{s) \\lds] ->■ 0, (4.15) 
Jo £ 

if e -> 0+. 

Finally applying (jPTty . fQ4l and (gJ5j> in (l4"T2"j) shows that 

p(e) -> 0, as e -> + . 

Hence from (14. lip and Gronwall's inequality we obtain 

sup E[\ V£ (t)\ 2 } ->0, 
te[o,r] 

as e — >■ + . ■ 

The following theorem contains our main variational equation, which is one of 
the main tools needed for deriving the maximum principle stated in Theorem 13.21 

Theorem 4.4 We suppose that (i) and (ii) in Theorem \3.2\ hold. For each e > 0, 
we have 

J(u e (-)) - J(u*(.)) = e E [MX*(T))p(T) ] 

+ eE[ [ £ x (X*(s),v*(s))p(s)ds] 
Jo 

+ E[ [ (£(X*(s),is e (s))-£(X*(s),u*{s)))ds} + o(e). (4.16) 
Jo 

Proof. We can write J{v E {-)) — J(v*(-)) as 

J(v £ (.))-J(v*(.)) = I 1 (e) + I 2 (e), (4.17) 

with 

h(e) = E[<j ) (X e (T))-<p(X*(T))} 

and 

h{e) =E[f (£(X £ (s), Ve (s)) - £(X*(s),u*(s))) ds}. 
Jo 
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Note that with the help of our assumptions and by making use of Lemma fl~3l 
Lemma 14.21 Lemma 14.11 and the dominated convergence theorem we deduce that 

-h(e) = -E[ / <j> x (X*(T) + 9(X e (T)-X*(T))(X e (T)-X*(T))de] 
e e Jo 

= E[ / MX*(T) + 9 (X £ (T) - X*(T)) (p(T) + Ve (T)) d6] 
Jo 

-+E[<j> x (X*(T))p(T)], as £^0 + . 

Hence 

h(s)=e E [ <j> x {X*{T))p{T) } + o{e). (4.18) 

Similarly 

- I 2 {e) = — E [ ( T (£(X E {s),v s (s)) - t(X*{s),u e (*))) ds) 
6 e Jo 

+ -E[f (£(X*(s),v £ (s))-£(X*(s),v*(s)))ds\ 
£ Jo 

= E[ f [ i x (X*(s) + 8(X e (s)-X*(s)),v £ (s))(p(s)+ Ve (s))d6ds] 
Jo Jo 

+ -E[f (£(X*(s),v £ (s))-£(X*(s),v*(s)))ds}. 
£ Jo 

On the other hand, applying Lemma 14.31 Lemma 14.21 Lemma 14.11 using the 
continuity and boundedness of £ x in (i), (ii) and the dominated convergence the- 
orem imply that 

E[ f [\ x (X*(s) + e(X £ ( S )-X*(s)) 1 u*(s)+Eu(s))(p( S ) + r ]£ ( S ))deds} 
Jo Jo 

-►E[ f £ x {X\s)y{s))p{s)ds\. 
Jo 

In particular we obtain 

hie) = eE[ I* £ x (X*(s),v*(s))p(s)ds] 
Jo 

+ E[[ (£(X*(s),is e (s))-£(X*(s),v*(s)))ds] + o(e). (4.19) 



As a result the theorem follows from (14.17[) . (14.181) and (I4.19p . 
Let us next introduce an important variational inequality. 
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Lemma 4.5 Let hypotheses (i), (ii) in Theorem \3.S\ hold. Then 

e E ( Y*(T),p{T) ) + e E [ C £ x (X*(s), u*(s)) p(s) ds] 

Jo 

+ E[ [ (6 £ n(s)-(5 £ b(s),Y*(s))-(5 £ a(s),Z*(s)) 2 )ds]>o(e), (4.20) 
Jo 

where 

5 e U(s) = U(X*(s),» e (s),Y*(s), Z*{s)) - n(X*(s), u*(s), Y* (s) , Z* (s)) . 

Proof. Since v*{-) is an optimal control, then J(i/ e (-)) — J(z/*(-)) > 0. Hence the 
result follows from (I4.16P and fl3.ll) . ■ 

The following duality relation between ( 14.1 ft and (j3.2j) is also needed in order 
to establish of proof of Theorem 13.21 

Lemma 4.6 Under hypothesis (i) in Theorem \3.B. we have 

E ( Y*(T),p(T) ) = — E[ f £ x (X*(s),v*(s))p(s)ds] 

Jo 

+ E[ [ (K{X*(s),u*(s))u{s) ,Y*{s))ds] 
Jo 

+ E[[ (o- u (X*(s),u*(s))u(s),Z*(s)) 2 ds]. (4.21) 

Jo 

Proof. The proof is done by using Yosida approximation of the operator A and 
Ito's formula for the resulting SDEs, and can be gleaned directly from the proof 
of Theorem 2.1 in [15]. ■ 

We are now ready to establish (or complete in particular) the proof of Theo- 
rem E2J 

Proof of Theorem HH Recall the BSEE (Q : 

- dY v U(t) = ( A* Y vi ~-\t) + V x n(X v U(t),v(t), Y u ^{t), Z v ^{t)) ) dt 

-Z»U(t)dW(t), < t < T, 

Y"W(T) = V0(X"W(T)). 

From Theorem 13 . 1 1 there exists a unique solution (Y*, Z*) to it. Thereby it remains 
to prove (13.41) . 
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Applying fH~20D and ffl~2TD gives 

E / ( 6 e H{s) + ( e K(X*(s),u*(s))u(s) - 5 E b(s) , Y*(s) > 



+ (e<j u (X*(s),v*(s))v(s)-6 e o-(s),Z*(s)) 2 ^ds} >o(s). (4.22) 

But, as done for (14.141) . by using the continuity and boundedness of b u in assump- 
tion (i) and the dominated convergence theorem, one can find that 

f-T 



- E [ f (e K(X*(s), v*(s))u(s) - S £ b(s) , Y*(s) ) ds 
£ Jo 

= -E[ f 
Jo 



[Y*(s), I [ K(X*(s),u*(s) + 9(u £ (s) - u*(s))) 

-b„(X*(s),u*(s)))p(s) d9)ds] ^0, 



as e — > + . This means that 

E[ / (sK{X*{s),u*{s))u{s)-8 e b{s),Y*{s))ds] = o{e). 







Similarly, 



E[ / (eo- u (X*(s),p*(s))v(s)-5 £ o-(s),Z*(s)} 2 ds) = o(e). 



o 



Now by applying these two former identities in (14.221) we deduce that 

E [ f S £ n(s)ds] > o(e). (4.23) 
Jo 

Therefore, by dividing (I4.23P by e and letting e — > + , the following inequality 
holds: 

e [ f (v u n(t, X*(t),u*(t), Y*(t), Z*(t)), u{t)) Q dt]>0. 
Jo 

Finally, (13 Ah follows by arguing, if necessary, as in [51 P. 280] for instance. ■ 
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